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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including tine fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on June 
23, 2009 has been entered. 

Information Disclosure Statement 

The information disclosure statement (IDS) submitted on June 23, 2009 is in 
compliance with the provisions of 37 CFR 1 .97. Accordingly, the information disclosure 
statement is being considered by the examiner. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 

Claims 60 - 61 , 67 - 68, 71 - 72, and 78 - 80 are rejected under 35 
U.S.C. 102(a) as being anticipated by Kresimir Matkovic et al. ("Visual Image Query", 
ACM, June 11 - 13, 2002, pages 116 - 123), hereinafter, "Matkovic." 
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With respect to claim 60, Matkovic teaches receiving a plurality of reference 
images (i.e. target images, or reference images are retrieved, page 1 17, 3: Main Idea, 
lines 1 - 3), calculating by a computer image parameters for each reference image 
wherein the image parameters characterize color, texture and shape features that are 
common to the reference image and at least one other reference image (i.e. parameters 
are calculated for reference and instant images, page 1 17, 3: Main Idea, lines 5-11), 
combining by the computer the calculated image parameters to generate a composite 
reference image (i.e. image parameters are reduced to generate a composite image, 
page 1 17, 3: Main Idea, lines 5 - 6), comparing by the computer the composite 
reference image to images in a collection in order to identify one or more of the images 
having features described by the composite reference image (input image is compared 
to other reference images based on calculated parameters, page 1 17, 3: Main Idea, 
lines 15 - 25). 

With respect to claim 61 , Matkovic teaches the reference images are ranked and 
wherein calculating parameters for each reference image includes weighting the 
parameters based on each reference image's rank (i.e. input image is compared to 
other reference images based on calculated parameters, page 117, 3: Main Idea, lines 
22-24). 

With respect to claim 67, Matkovic teaches calculating image parameters for the 
one or more images in the collection (i.e. image parameters are calculated for all 
images in a collection, page 1 17, 3: Main Idea, lines 1 - 3). 
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With respect to claim 68, Matkovic teaches comparing the composite reference 
image to parameters of each of the one or more images in the collection (i.e. query 
image (composite image) compared to images in collection, page 1 17, 3: Main Idea, 
lines 15-17). 

With respect to claim 71 , Matkovic teaches receiving a plurality of reference 
images (i.e. target images, or reference images are retrieved, page 1 17, 3: Main Idea, 
lines 1 - 3), calculating by a computer image parameters for each reference image 
wherein the image parameters characterize color, texture and shape features that are 
common to the reference image and at least one other reference image (i.e. parameters 
are calculated for reference and instant images, page 1 17, 3: Main Idea, lines 5-11), 
combining by the computer the calculated image parameters to generate a composite 
reference image (i.e. image parameters are reduced to generate a composite image, 
page 1 17, 3: Main Idea, lines 5 - 6), comparing by the computer the composite 
reference image to images in a collection in order to identify one or more of the images 
having features described by the composite reference image (input image is compared 
to other reference images based on calculated parameters, page 1 17, 3: Main Idea, 
lines 15 - 25). 

With respect to claim 72, Matkovic teaches reference images are ranked and 
wherein calculating parameters for each reference image includes weighting the 
parameters based on each reference image's rank (i.e. input image is compared to 
other reference images based on calculated parameters, page 117, 3: Main Idea, lines 
22 - 24). 
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With respect to claim 78, Matkovic teaches calculating image parameters for the 
one or more images in the collection (i.e. image parameters are calculated for all 
images in a collection, page 1 17, 3: Main Idea, lines 1 - 3). 

With respect to claim 79, Matkovic teaches comparing the composite reference 
image to parameters of each of the one or more images in the collection (i.e. query 
image (composite image) compared to images in collection, page 1 17, 3: Main Idea, 
lines 15-17). 

With respect to claim 80, Matkovic teaches one or more computers operable to 
execute the program product, interact with the display device, and perform operations 
comprising receiving a plurality of reference images (i.e. target images, or reference 
images are retrieved, page 117, 3: Main Idea, lines 1 - 3), calculating by a computer 
image parameters for each reference image wherein the image parameters characterize 
color, texture and shape features that are common to the reference image and at least 
one other reference image (i.e. parameters are calculated for reference and instant 
images, page 1 1 7, 3: Main Idea, lines 5-11), combining by the computer the calculated 
image parameters to generate a composite reference image (i.e. image parameters are 
reduced to generate a composite image, page 117, 3: Main Idea, lines 5 - 6), 
comparing by the computer the composite reference image to images in a collection in 
order to identify one or more of the images having features described by the composite 
reference image (input image is compared to other reference images based on 
calculated parameters, page 117, 3: Main Idea, lines 15 - 25). 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary sl^ill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 62 - 64, and 73 - 75 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kresimir Matkovic et al. ("Visual Image Query", ACM, June 11-13, 
2002, pages 1 16 - 123) in view of Essam El — Kwae et al. ("A Robust Framework for 
Content - Based Retrieval by Spatial Similarity in Image Databases," ACM, 1999, 
pages 174 - 198), hereinafter, "El - Kwae." 

With respect to claim 62, Matkovic teaches a visual image query method 
(Introduction, page 116, lines 36 - 39). Matkovic does not explicitly teach raster data as 
claimed. 

However, El-Kwae teaches a reference image includes raster data (i.e. reference 
image includes shape data, page 175, line 7). 

Matkovic and El - Kwae are analogous art because they are from the same field 
of endeavor of image searching. At the time of the invention, it would have been 
obvious to one of ordinary skill in the art to modify the teachings of Matkovic with the 
teachings of El - Kwae in order to search for and retrieve images using spatial similarity 
(El - Kwae, page 174, lines 1 - 5). 
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With respect to claim 63, Matkovic teaches a visual image query method 
(Introduction, page 116, lines 36 - 39). Matkovic does not explicitly teach vector data as 
claimed. 

However, El-Kwae teaches a reference image includes vector data (i.e. images 
have vector data, page 181 , line 30). 

Matkovic and El - Kwae are analogous art because they are from the same field 
of endeavor of image searching. At the time of the invention, it would have been 
obvious to one of ordinary skill in the art to modify the teachings of Matkovic with the 
teachings of El - Kwae in order to search for and retrieve images using spatial similarity 
(El - Kwae, page 174, lines 1 - 5). 

With respect to claim 64, Matkovic teaches a visual image query method 
(Introduction, page 116, lines 36 - 39). Matkovic does not explicitly teach audio objects 
as claimed. 

However, El-Kwae teaches the collection includes audio objects (i.e. may be 
implemented with audiovisual search engines, page 178, line 15)). 

Matkovic and El - Kwae are analogous art because they are from the same field 
of endeavor of image searching. At the time of the invention, it would have been 
obvious to one of ordinary skill in the art to modify the teachings of Matkovic with the 
teachings of El - Kwae in order to search for and retrieve images using spatial similarity 
(El - Kwae, page 174, lines 1 - 5). 
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With respect to claim 73, Matkovic teaches a visual image query method 
(Introduction, page 116, lines 36 - 39). Matkovic does not explicitly teach raster data as 
claimed. 

However, El-Kwae teaches a reference image includes raster data (i.e. reference 
image includes shape data, page 175, line 7). 

Matkovic and El - Kwae are analogous art because they are from the same field 
of endeavor of image searching. At the time of the invention, it would have been 
obvious to one of ordinary skill in the art to modify the teachings of Matkovic with the 
teachings of El - Kwae in order to search for and retrieve images using spatial similarity 
(El - Kwae, page 174, lines 1 - 5). 

With respect to claim 74, Matkovic teaches a visual image query method 
(Introduction, page 116, lines 36 - 39). Matkovic does not explicitly teach vector data as 
claimed. 

However, El-Kwae teaches a reference image includes vector data (i.e. images 
have vector data, page 181 , line 30). 

Matkovic and El - Kwae are analogous art because they are from the same field 
of endeavor of image searching. At the time of the invention, it would have been 
obvious to one of ordinary skill in the art to modify the teachings of Matkovic with the 
teachings of El - Kwae in order to search for and retrieve images using spatial similarity 
(El - Kwae, page 174, lines 1 - 5). 
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With respect to claim 75, Matkovic teaches a visual image query method 
(Introduction, page 116, lines 36 - 39). Matkovic does not explicitly teach audio objects 
as claimed. 

However, El-Kwae teaches the collection includes audio objects (i.e. may be 
implemented with audiovisual search engines, page 178, line 15). 

Matkovic and El - Kwae are analogous art because they are from the same field 
of endeavor of image searching. At the time of the invention, it would have been 
obvious to one of ordinary skill in the art to modify the teachings of Matkovic with the 
teachings of El - Kwae in order to search for and retrieve images using spatial similarity 
(El - Kwae, page 174, lines 1 - 5). 

Claims 65 - 66, and 76 - 77 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kresimir Matkovic et al. ("Visual Image Query", ACM, June 11-13, 
2002, pages 1 16 - 123) in view of Essam El — Kwae et al. ("A Robust Framework for 
Content - Based Retrieval by Spatial Similarity in Image Databases," ACM, 1999, 
pages 174 - 198) in view of Howard Wactlar et al. (U.S. Patent 5,835,667), hereinafter, 
"Wactlar." 

With respect to claim 65, Matkovic teaches combining further comprises 
receiving one or more reference objects (i.e. target images, or reference images are 
retrieved, page 1 17, 3: Main Idea, lines 1 - 3), and El - Kwae teaches searching and 
retrieving images using special similarity (page 174, lines 1 - 5). The combination of 
Matkovic and El - Kwae does not explicitly disclose calculating and searching sound 
and video objects as claimed. 
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However, Wactlar teaches calculating sound parameters for each reference 
audio object wherein the sound parameters characterize sound features in the 
reference object that map to image parameters of at least one reference image (i.e. 
audio data is transcribed and indexed, column 4, lines 34 - 41), combining the sound 
parameters with the Image parameters to produce composite reference information (i.e. 
segmented audio/video data is combined, column 6, lines 45 - 54). 

Matkovic, El - Kwae, and Wactlar are analogous art because they are from the 
same field of endeavor of image searching. At the time of the invention, it would have 
been obvious to one of ordinary skill in the art to modify the teachings of Matkovic and 
El - Kwae with the teachings of Wactlar in order to improve video searching capabilities 
(column 4, lines 9-16). 

With respect to claim 66, Matkovic teaches a visual image query method 
(Introduction, page 116, lines 36 - 39), and El - Kwae teaches searching and retrieving 
Images using special similarity (page 174, lines 1 - 5). The combination of Matkovic and 
El - Kwae does not explicitly disclose calculating and searching sound and video objects 
as claimed. 

However, Wactlar teaches comparing the composite reference Information to 
Images and sound objects In a collection in order to identify one or more of the Images 
or the sound objects having features described by the composite reference Information 
(i.e. images, videos, and sounds are compared to find similar objects, column 12, lines 
33 - 40). 
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Matkovic, El - Kwae, and Wactlar are analogous art because they are from the 
same field of endeavor of image searching. At the time of the invention, it would have 
been obvious to one of ordinary skill in the art to modify the teachings of Matkovic and 
El - Kwae with the teachings of Wactlar in order to improve video searching capabilities 
(column 4, lines 9-16). 

With respect to claim 76, Matkovic teaches receiving one or more reference 
audio objects (i.e. target images, or reference images are retrieved, page 1 17, 3: Main 
Idea, lines 1 - 3), and El - Kwae teaches searching and retrieving images using special 
similarity (page 174, lines 1 - 5). The combination of Matkovic and El - Kwae does not 
explicitly disclose calculating and searching sound and video objects as claimed. 

However, Wactlar teaches calculating sound parameters for each reference 
audio object wherein the sound parameters characterize sound features in the 
reference sound object that map to image parameters of at least one reference image 
(i.e. audio data is transcribed and indexed, column 4, lines 34 - 41), combining the 
sound parameters with the image parameters to produce composite reference 
information (i.e. segmented audio/video data is combined, column 6, lines 45 - 54). 

Matkovic, El - Kwae, and Wactlar are analogous art because they are from the 
same field of endeavor of image searching. At the time of the invention, it would have 
been obvious to one of ordinary skill in the art to modify the teachings of Matkovic and 
El - Kwae with the teachings of Wactlar in order to improve video searching capabilities 
(column 4, lines 9-16). 
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With respect to claim 77, Matkovic teaches a visual image query method 
(Introduction, page 116, lines 36 - 39), and El - Kwae teaches searching and retrieving 
images using special similarity (page 174, lines 1 - 5). The combination of Matkovic and 
El - Kwae does not explicitly disclose calculating and searching sound and video objects 
as claimed. 

However, Wactlar teaches comparing the composite reference information to 
images and sound objects in a collection in order to identify one or more of the images 
or the sound objects having features described by the composite reference information 
(i.e. images, videos, and sounds are compared to find similar objects, column 12, lines 
33 - 40). 

Matkovic, El - Kwae, and Wactlar are analogous art because they are from the 
same field of endeavor of image searching. At the time of the invention, it would have 
been obvious to one of ordinary skill in the art to modify the teachings of Matkovic and 
El - Kwae with the teachings of Wactlar in order to improve video searching capabilities 
(column 4, lines 9-16). 

Claims 69 - 70 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Kresimir Matkovic et al. ("Visual Image Query", ACM, June 11 - 13, 2002, pages 116- 
123) in view of Howard Wactlar et al. (U.S. Patent 5,835,667), hereinafter, "Wactlar." 
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With respect to claim 69, Matkovic teaches receiving a plurality of reference 
objects (i.e. target images, or reference images are retrieved, page 1 17, 3: Main Idea, 
lines 1 - 3). 

Matkovic does not explicitly disclose that the reference objects are video objects. 

However, Wactlar teaches calculating by a computer video parameters for each 
reference video object wherein the video parameters characterize image and audio 
features in the reference video object that are common to the reference video object 
and at least one other reference video object (i.e. audio data is transcribed and indexed, 
column 4, lines 34 - 41 ), combining by the computer the video parameters to generate 
composite reference information (i.e. segmented audio/video data is combined, column 
6, lines 45 - 54), comparing by the computer the composite reference information to 
video objects in a collection in order to identify one or more of the video objects having 
features described by the composite reference information (i.e. images, videos, and 
sounds are compared to find similar objects, column 12, lines 33 - 40). 

Matkovic and Wactlar are analogous art because they are from the same field of 
endeavor of similarity searching. 

At the time of the invention, it would have been obvious to one of ordinary skill in 
the art to modify the teachings of Matkovic with the teachings of Wactlar in order to 
improve video searching capabilities (column 4, lines 9-16). 

With respect to claim 70, Matkovic teaches receiving a plurality of reference 
objects (i.e. target images, or reference images are retrieved, page 1 17, 3: Main Idea, 
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lines 1 - 3). Matkovic does not explicitly disclose that the reference objects are text 
objects. 

However, Wactlar teaches calculating by a computer text parameters for each 
reference text object wherein the text parameters characterize language features in the 
reference text object that are common to the reference text object and at least one other 
reference text object (i.e. textual parameters are calculated, column 6, lines 39 - 54), 
combining by the computer the text parameters to generate composite reference text 
(i.e. text, audio, and video are combined for similarity matching, column 7, lines 21 - 34), 
and comparing by the computer the composite reference text to text objects in a 
collection in order to identify one or more of the text objects having features described 
by the composite reference text (i.e. full text search is performed, column 5, lines 14 - 
24). 

Matkovic and Wactlar are analogous art because they are from the same field of 
endeavor of similarity searching. 

At the time of the invention, it would have been obvious to one of ordinary skill in 
the art to modify the teachings of Matkovic with the teachings of Wactlar in order to 
improve video searching capabilities (column 4, lines 9-16). 
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Conclusion/Contact Information 

Any inquiry concerning this communication or earlier communications from tine 
examiner should be directed to ALEXANDRIA Y. BROMELL whose telephone number 
is (571 )270-3034. The examiner can normally be reached on M - R 9 - 3. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John R. Cottingham can be reached on 571-272-7079. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 

Patent Application Information Retrieval (PAIR) system. Status information for 

published applications may be obtained from either Private PAIR or Public PAIR. 

Status information for unpublished applications is available through Private PAIR only. 

For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 

you have questions on access to the Private PAIR system, contact the Electronic 

Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 

USPTO Customer Service Representative or access to the automated information 

system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Alexandria Y Bromell/ /Shahid Al Alam/ 

Examiner, Art Unit 2167 Primary Examiner, Art Unit 2162 

July 14, 2009 



